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The purpose of this project was to provide 
information en the appropriate uses of educational tests and 
interpretations of test results to citizen groups. The major 
audiences were parents, legislators, school board members, lawyers, 
journalists and others who would be interested in and profit from 
training related to educational testing. Nero and Associates, Inc., 
together with the Northwest Begional Educational Laboratory and the 
California Department of Education, arranged, designed and conducted 
five seminars/workshops for groups and organizations representing the 
above audiences. The format and specific content for each 
seminar/workshop was developed in collaboration with representatives 
from each group. For example, lawyers were involved in designing the 
workshop for attorneys to insure that the information presented would 
address issues ,of concern to that group. Staff from the National 
Institute of Education worked with the contractor to establish the 
workshop agendas and identify the key groups. The workshops were held 
from April 19B0 through September 19B0. The work resulted in a set of 
training materials which can be used by measurement specialists to 
train citizen groups. The materials are available through NIE. 
(Author/Bl) 
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FINAL REPORT 
INVITATIONAL TESTING SEMINARS FOR NOH-EDUCATORS 
NATIONAL INSTITUTE OF EDUCATION 
CONTRACT #400-79-0059/SB0408 (a)-79-C-197 

Nero and Associates, Inc. submits herewith the Final Report on 
Contract #400-79-0059/SBO408 (a)-79-C-197 to the National Institute of 
Education (NIE) . This Final Report contains the background cmd a brief 
summary of the project, a summary of participemt reactions, a section on 
recommended dissemination activities and a project abstract. 

Background 

In Mcurch 1978, the U. S. Department of Health, Education and Welfare 
sponsored a National Conference on Achieveme:it Testing and the Basic Skills. 
This conference brought together over 300 educators, educational adminis- 
trators, testing and measurement experts, government officials and 
representatives of parents' and teachers' organizations to explore the 
role of achievement testing in education and the role of the Federal 
govemment in achievement testing. As a direct response to recommendations 
from the conference , NIE initiated a number of activities in the general 
area of dissemination of information about achievement testing. 

As part of this effort, NIE contracted with Nero and Associates, Inc. 
to develop, arrange for and conduct five seminars/workshops on educational 
testing issues for audiences interested in, but not directly involved 
in, the education process, i.e., parents, legislators, lawyers, school 
board members and journalists. The purpose of the seminars and workshops 
was to provide participants with current information on (1) test 
characteristics, uses and issues which affect the interpretation and 
reporting of test results, and (2) the legal and social issues stjurrounding 



testing. Nero and Aaaociates sub-contracted with the Northwest Regional 
Educational Iiaboratory of Portland, Oregon, and the California State 
Depeurtment of Education for the development and preparation of the 
seminar /workshop training materials and the conduct of the seminars/ 
workshops. 

Project Summary 

The primary tasks accomplished during the life of the project 
involved : 

o Arranging, developing materials for and conducting five seminars 
and workshops on educational testing issues for parents , 
legislators, lawyers, school board members and journalists; and 

o Developing a set of training materials for use by other trainers 
v/ho are involved in conducting seminars /workshops on testing. 

Activities included: 

1. Identifying cmd contacting potential audiences and arranging for 
their participation 

Project staff, in collaboration with NIE, selected 

organizations which would (1) represent the appropriate audience 

groups mentioned above, cind (2) have national or regional 

meetings scheduled to occur within the timeframe established 

for the contract. Although many of the organizations expressed 

considerable interest in the seminar, almost all were so far 

along in the planning process that it was not feasible to 

offer the NIE testing seminar as a pre- or post-session. 

However / in two cases project staff were able to schedule the 

seminar in conjiinction with existing meetings (the seminars 

for parents and lawyers) . 
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Project staff arranged for the other three seminars/ 
workshops (for school board members, joiirnali;its and legislators) 
by contacting specific target groups dirtictly (e.g., state school 
board associations, state legislatures, local media, etc,) to 
determine their interest in and willingness -lo sponsor the 
testing seminars /workshops for their members. Once identified, 
project staff worked directly with representatives of each of 
the groups to design a seminar/workshop tailored to meet the 
particular needs of the group in temE of content ar.d format- . 
The five seminars /workshops arranged for and conducted dviring 
this project are shown in Table 1- 

Selecting and preparing appropriate training materials to address 
the needs of each group 

During the initial months of the contract, project staff 

conducted a review of existing training materials in the area 

of testing. As specific groups were identified and indicated 

their willingness to participate in the seminar, it became 

increasingly apparent that much of the material needed for each 

seminar would have to be specially developed. As organizations 

identified general topics within the framework of the legal 

and social issues they would like to have covered in their 

seminar, project staff developed presentation outlines, 

handouts, transparencies and a resoxarce guide for each topic 

area. The presentation outlines, handouts and transparencies 

then became part of Training Citizen Groups on Educational 

Testing Issues: A Trainer's Manual; the resovirce materials 



TABLE 1 

Invitational Testing Seminars for Non-Educators 



TARGET GROUP 


AUDIENCE 


DATE AND PLACE 


APPROXIMATE 
NO. OF PARTICIPANTS 


TyPE OF SESSION 


Parents 


Region III Coalition of 
Title I Parents 


April 12, 1980 
Baltimore Hilton Inn 
Baltimore, MD 


90 


Three 1 hour and 15 
minute sessions during 
scheduled Inservice 
Training Meeting 


Legislators 


Ohio State Legislature's 
House and Senate Educa- 
tion Committees 


June 18, 1980 
Sheraton-Columbus Hotel 
Columbus, OH 


25 


Special session in 
evening (/:30 pj.- 
10:00 p.m.) 


School Board Hembers 


Ohio School Boards 
Association 


June 20, 1980 
Carrousel Inn 
Columbus, OH 


85 


All -day session 
(9:00 a.m.-4:00 p.m.) 


School Board Members 
and Journalists 


San Francisco Bay Area 
School Board Members 
and Journalists 


July 12, 1980 
Airport Hilton Hotel 
San Francisco, CA 


30 


All-day iovitational 
session held on a 
Saturday (8:45 a.m.- 
3:30 p.m.) 


Lawyers 


Center for Law and 
Education (Legal Services 
Corporation Training 
Session for Lawyers, 
Para-legals and clients) 


September 2!i, 198v 
Leamington Hotel 
Oakland, C/i 


75 - 80 


All-day post-session 
to already scheduled 
conference (9:00 a.m.- 
3:45 p.m.) 
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in expanded form are now part of two booklets entitled 
A Parent's Guide to Test in g in the Schools and Educational 
Facta and Issues i A Layperson's Guide to Testing in the 
Schools » 

Throughout the duration of the project, information 
contained in each of the three products was upcjated and 
revised based on (1) informal feedback provided by the 
seminar /workshop participants, (2) formal critiques provided 
by reviewers selected by project staff, (3) formal critiques 
provided by staff from NIE and NIE-selected reviewers , and 
(4) the cummulative experience of project staff as they worked 
with each of the sponsoring agencies in conducting the seminars/ 
workshops • 

Scheduling and arranging five local pilot workshops to try out 
the materials and workshop formats 

As part of the development process, project staff 

contacted local persons whose roles were similar to those of 

the actual participants. Separate small-scale pilot workshops 

were conducted in which both the format of the seminar /workshop 

and the content of the materials were tried out cind critiqued. 

Information provided by the participants in these pilot 

workshops was then used to refine the presentations and the 

products before conducting the actual seminars /workshops. 

Making all logistical and support arrangements for the seminars/ 
workshops 

Project staff worked closely with designated representatives 
from each organization in arranging the meeting space, meals, 
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invitational letters i ordaring n^Q^Qsary oquipmenfci 4m1 
gonarftlly facilitating all the logiatiaal and aiipport 
requirements for ©aoh seminar /workahop. 
Conduoting the workahopa 

Aa shown in Table 1, the five aaminara /workshops were held 
between April and September 1980, The number, of prasentera 
at each seminar /workshop varied from two to three depending on 
the size of the seminar /workshop and the number of special 
iaaueb detenninod collaboratively with the bponaoring agency. 
Onsite logistical and support arrangements were carried out 
by a representative from Nero and Associates. Overall, 
approximately 310 persons participated in the five seminars/ 
workshops; of these, 164 completed the participant response 
form (see the section on Summary of Participant Reactions to 
the Seminars /Works hops for a discussion of the data). 
Reporting to NIE on the workshops 

Project staff submitted monthly progress reports to NIE 
detailing all of the steps involved in developing the materials 
and conducting the seminars /workshops. In addition, periodically 
throughout the contract, project staf, u reported to and 
worked collaboratively with the NIE project monitor. Following 
each seminar /works hop. project staff submitted the participant 
reaction forms to NIE for informational purposes. Copies of 
all d\<xi.t materials, invitational letters, letters received by 
project staff in response to the seminars /workshops , etc., 
were forwarded to NIE on a regular basis. 
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Th« NIB projtiiat'. offiuav ^tt^dmleii ^ftah of tha fiva 

»«min4r«/workahopa 4mt, ciuirlna thasa aaaaluna, mat with 

ataff to review materlAla ^nd pvogreaa to data. 

7 , Delivering finaX reviaad varaiuna ot? tl^a wurkahcn) mAtar.UX&i 
for dlaeeminfttion 

Following the raviaw procasa daacribed in activity #2 

above, project staff proceeded with the final reviaion and 

production of the training materials. Aa specified in the 

contract, 10 copies of each of the three products were aubmitted 

to NIE along with the camera-ready copy for each. In addition, 

Nero and Associates prepared a final project £d)stract for NIE's 

submission to ERIC. Recommendations concerning the dissemination 

of the seminar /workshop materials are discussed in the final 

section of this report. 

Summary of Participant Reactions to the Seminars /Workshops 

Participant reaction to the seminars /workshops was overwhelmingly 
enthusiastic. A total of 164 participant response forms were received 
for the five seminars /workshops. Of these, 132 participants (80 percent) 
responded with an emphatic "Yes" to the question, "Do you think others 
would find the workshop helpful? See Appendix A for summary of comments. 

The most frec[uent comment on the participant response forms had to 
do with allowing more time for the seminars /workshops. The amount of 
time allocated for the seminars /workshops was determined largely by the 
sponsoring group. This accomts in part for the variation in length of 
the sessions (see Table 1). At the conclusion of the five seminars/ 
workshops (which ranged in time from 1 hour aind 15 minutes to 7 hours) , 



E4cih of tha aamln^rft/workahopa hAd a tUCtarent format: which was 
dt^vt^lopad to meet the needs of the Individual rtiidlennefci. The «enilnrtj.' for 
p«;rents w^e of Cared 4t th^ree aoneeaut lve one-^hour and 15 minute eeea tona 
AS p«rt of ^n Inaervice Training Meeting, The aaminnr for leglalAtora 
WA« A 2 And ona-hAlf hour evening sesaion following a reception and dinner, 
Tha school board workshop and the combined school board/ journalist workshop 
War« both all^day acaaionc, with a cholrt' In tho /afternoon of « q^r\f^rr)} 
simulation exercise or a presentation and discussion of issue-oriented 
topics. The workshop for lawyers was also an all-day session with a 
simulation exercise in the morning and a choice of three "issue'* workshops 
in the afternoon. 

Participants were asked to assign an overall rating to the seminar/ 
workshop they attended (poor rating»l , excellent rating»5). In a 
comparison of the five seminars /workshops , the percentage of participants 
who found the workshop they attended to be good to excellent (4-5 on the 
scale) is as follows: 



Seminar /Workshop 



Percentage of 
Respondents 



1. 



School Board Members and Journalists 



100% 



2. 



Legislators 



93% 



3. 



School Board Members 



88% 



Parents 



78% 



5. 



Lawyers 



58% 
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School Bocxd Members and Journalists ^ San Francisco 

On the basis of overall ratings of the seminars /workshops , 
the workshop for school boeurd members and joiamalists held in 
San Francisco was the most successful. The 5i-hour Saturday work 
session covered the major legal issues in testing, a simulation 
activity emd future trends in testing. 

Some of the responses to the question, "What did you like 
most about: the workshop?" reveal what- participants gained from 
the workshop: 

. •the simulation activity offered an opportunity to 
interact in small groups, expanding on items covered in 
the morning session," 

"the workshop did more than answer questions; it raised 
issues and helped me become aware of what I need to know. " 

Ninty percent of the participants would reconunend the 
workshop to others, especially to teachers, administrators and 
school board members. 
Legislators , Ohio 

The seminar for legislators, a 2i-hour session with presentations 
on minimum competency, "truth in testing" and general testing 
issues, was rated good or excellent by 93 percent of the 
participants . 

Some of the reasons for favorable response are summarized 

in the following participant comments: 

"Competency testing is a burning issue. . .it is helpful 
to find out that competency testing programs are working 
in other states." 
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••I welcomed the broad stress on the policy implications 
of legislative decisions. 

Eighty-six percent of the participants thought the seminar 
would be helpful to other legislators. 
School Board Members » Ohio 

This school bofiurd member workshop was cm all-day session 
with a format similar to that xised in San Francisco. Eighty-eight 
percent of the paurticipants rated the workshop as good or excellent. 

Some aspects Ohio school board members liked about the 
workshop were: 

••Si^port materials in the packet are appreciated." 

". . .content was appropriate." 

. .the open atmosphere of presentation and receptiveness 
of opinions was gratifying." 

". . .learned a lot about trends in other states, including 
litigation. " 

Eighty -three percent of the school board members felt others 
would benefit from this workshop. Parents, teachers and 
administrators were suggested as possible audiences. 
Parents , Baltimore 

The seminar for parents, held in Baltimore, consisted of a 
one hour and 15 minute session repeated three times. This format 
was chosen to accommodate the overall format of the training 
sessions established by the sponsoring group. The parent seminar 
was designed to answer general questions parents might have about 
testing. Due to the brevity of each session, many of the 
participants commented that they would like the seminar 
to be longer. 
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The areas that pcurents particularly liked were described 

in their responses: 

"All the information was geared to what parents are really 
trying to find out about the testing of their children." 

. .that scores are not the most important aspect of 
testing. " 

(As a result of the seminar) "parents will know more 
about the tests given in schools." 

Seventy-three percent of the participants thought other 
peurents would find the seminar helpful. 
Lawyers , Oaklemd 

The workshop for lawyers was held as a full-day post-session 
to an already scheduled three-day training session for lawyers, 
pcuralegals and parent and student lay advocates. The agenda 
included a simulation activity in the morning and three concurrent 
sessions on cult\aral bias, competency testing and hints for 
helping parents and stiadents cope with testing which were offered 
twice in the afternoon. 

Responses to this workshop varied. The participant response 
form for this workshop differed from the one used for the other 
semincurs eind workshops in that pcurticipants were asked to react 
to the morning's simulation activity as well as to the afternoon 
issue sessions. 

Comments received included: 

"The simulation activity was very good in that it provided 
some direction for practical application." 

"Information on how to legally address cultural bias in 
testing would be valuable." 
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• •the workshop gave me the sensitivity to cultural 
bias necessary to advocate on behalf of students and groups 
being tested. I did not agree with all of the presenter's 
interpretations, but the premise and theories were sound." 

"Presenters' discussion on legal challenges to minimum 
competency testing. . .most useful. . • .What it lacked 
was a political framework to address why these issues are 
emerging, " 

Fifty-eight percent of the participants rated this workshop 
as good to excellent. However, 38 percent rated it as only fair 
in their overall rating of the workshop. In reviewing the comments 
on the forms rated "fair," many of tne participants wanted more 
material presented to further their advocacy goals. This is 
consistent with their earlier training in that participants had 
spent three days being trained to be advocates in the is of 
special education and student discipline. However, the purpose of 
the NIE workshop was to explain general educational testing 
practices as well as current issues in testing rather than to 
advocate from a particular perspective. Thus, some participants 
felt the NIE workshop failed to provide advocacy skills and a 
political framework to analyze these issues. Their ratings 
reflected their dissatisfaction. 

Overall, however, 85 percent of the respondents thought others 
would find the workshop helpful. 

Dissemination Recommendations 

Since one of the primary purposes for the project was conducting 
five seminars /workshops, this made dissemination an integral part of 
the work. However, it is important to think in terms of what 
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dissemination activities might occur to promote the use of the 
products. We will discviss such activities in the following two 
sections: Non-NIE dissemination and NIE dissemination. 
Non-NIE Dissemination 

The contract ended on September 28, 1980. However, project 
staff feel strongly that audiences should have an opportunity to 
experience the training which others received this year and to use 
the materials developed as a result of the contract. Thus, 
although no official dissemination is- currently budgeted or 
planned, some dissemination activities will, in all likelihood, 
tcike place. Some examples are: 

1. Project staff intend to present information about the 
project — its processes, products and outcomes — at the 
National Council on Measurement in Education annual meeting 
in the spring of 1981. The purpose will be to acquaint 
measurement specialists with the Trainer's Manual and 

the related training products so that the specialists 
might use these products in their work with groups 
interested in issues surrounding educational testing. 

2. Project staff will respond to requests for training on 
an "as-needed" basis. Every attempt will be made to 
keep client costs low and to use the materials already 
developed. 

3. The Northwest Regional Educational Laboratory will take 
the following steps to promote the materials: 

o Place the training materials in the ERIC system. 

o Announce in the CAPT Newsletter and the Laboratory's 
newsletter ( The Northwest Report ) that the materials 
are available through the ERIC system and by writing 
to NIE. 

4. Project staff will forward all inquiries concerning the 
availad^ility of materials to NIE for follow-up. Since 
NIE will have the camera-ready copy, this appears to be 
the most feasible response. 
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5. The California State Department of Education has 

already planned further workshops in collaboration with 
the California School Boards Association using the 
workshop materials. 

NIE Dissemination 

The primary dissemination responsibility rests with NIE. 
Should NIE desire to promote the seminars /workshops and/or the 
training materials, project staff would suggest the following 
kinds of activities: 

1. NIE could contract for additional work in this area. 

The seminars/workshops, conducted to date were considered 
to be extremely useful by those who attended. In addition, 
participants recommended that the seminars be offered to 
others. Now that the materials have been developed, a 
cadre of trainers available and the logistics of offering 
such seminars /workshops worked out, NIE could choose to 
continue the current project in a modified form. The 
audiences to date expressed strong support for NIE making 
this service available to them. There is qvery reason 
to assume that additional activities in this area would 
be received in a similar fashion. 

2. NIE could promote the use of the materials through regular 
mailings to appropriate audiences. Through the use of 
already-established channels of communication, NIE could 
share information about the products with both those who 
might serve as trainers and those who might welcome such 
training. 

3. NIE could publish information about the project and 
materials in the NIE products catalog. In this way, those 
who receive the catalog will at least know that the work 
has been conducted and have an opportunity to contact NIE 
should they desire further information. 

4. NIE could disseminate information about the products to 
related NIE-funded projects. The projects funded through 
the Testing, Assessment and Evaluation Unit within NIE 
may have a particularly high interest in the products 
resulting frcan this contract. NIE might make copies of 
the products available to Principal Investigators who 
are working in the general area of testing. 



5. NIE could offer small grants to groups who would like 
to receive training • For a small amovait of money, NIE 
could offer to support expsmded use of the products 
through finemcing seminars /workshops on a grsmt basis, 
Groiips such as parent orgeoiizations, journalists, etc. 
would then have the opportunity to apply for funds which 
they could then use to purchase the training, 

6. NIE could print the products, using the Government 
Printing Office (GPO) , and make additional copies 
available. Selected products could then be distributed 
to likely trainers (e.g.. Labs and Centers) and/or 
likely audiences (e.g., the PTA) for their use. Audiences 
could, at the same time, be referred to the present 
contractor should they wish to purchase training. 

7. NIE could send an annovmcement and suggested ways to 
make use of the materials to each State Department of 
Education. 

8. NIE could send an amnouncement cibout the materials 
being available to professional associations (and parent 
groups) representing the target audiences and suggest 
resources as a meams of putting the seminars /workshops on. 

9. NIE could speak to professional association meetings 
concerning the project and the products. In this way, 
NIE could expand the awareness of the products and let 
audiences know that both the products and the training 
are availcd^le. 

10. NIE could submit the Parent's Guide and Layperson's Guide 
to CIC (Curriculum Information Center) and other resource 
agencies for announcement of availability. 



Conclusion 

NIE and project staff can both use existing mechanisms to promote 
awareness about the products and the availability of training. The 
examples cited above are meant to be illustrative of this process. 
The most direct way of ensuring the use of the products — and an 
increase in knowledge about testing on the part of lay groups — would 
be to contract for specific dissemination training. 
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SUMMARY OF PARTICIPANT REACTIONS 

Seminar for Parents 

April 12, 1980 
Baltimore, Maryland 

What changes to you think we should make in the workshop? (For example, 
ot her points that should be covered, points that were unclear, or other 
styles of presentation.) (Total n=42) 

Session #1 (n=32) 

More time. 

Leave questions until last. 

You need more time for your workshop. Need to give more information on 
questions being asked. 

.^Presentation fine. Not enough time; your workshop should be longer. 
Really tell parents point-blank how they need to help their child. 
Keep it the same please. 

Not enough time. Being from different area not enough time to ask questions. 

The presentation was good. The time was too short. 

We need more time to cover different points. 

I feel that most important issues were discussed. 

Nonel Just allow more time for discussion after presentation. 

The workshop was well prt^sented with ample time for questions between 
each subject. 

Discontinue questionnaire. Valuable time is used for such a short 
session. We could have heard more. 

Deal with the exact tests used in the school district. Need more time. 

More time was needed in order to cover the subjects. 

How can you tell if a test is a good test? More information on teachers' 
and parents* role during testing periods. 

Present all materials and information before hand. 
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Closer control as the questions and discussions tended to stray and two 
or three persons monopolized the session. 

Give more time. 

Have presenters give the objectives at the beginning of the presentation. 
Have presenters give more detailed explanation before answering individual 
concerns and selfish interests. 

Overall discussion was good but insufficient amount of time to answer 
questions. 

Workshop should be longer. 

I did not learn about testing at all. 

Most parents are familiar with test procedures. More time should be 
spent hearing parents* views. 

There should be more time on evaluation. 

Points that were unclear. 

Shorten the subject matter and get deeper into it. Limit questions. 

Explain more, so parents could get a better understanding. Instead of 
parents asking questions. 

Parents weren't given enough points on how to help children prepare for a 
test. 

I don't think this particular audience was interested in generalized 
understanding of tests. They want to know about their children's 
situations. I think you have to use simpler language. Also, for this 
kind of group you had too many participants. You should really limit 
it to fifteen. 

Do a group assessment by reading your questions and having members raise 
their hands. This could be quicker and just as accurate. For each 
questions give a 1, 2, 3, 4, for content and then give time for 
questions. Keep control of your group. 

Points were unclear and not explained to the questions asked. Some 
.tests should be named and explained especially the types of tests used 
in the Title I programs around the U.S. 

Session #7: (n=5 ) 

Demonstration materials for parents to actually use learned information, 
sample tests to view, etc. 

Form chairs in semi-circle— better voice pitch and eye contact for 
parent participants. 

22 
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I think an explanation of different kinds of tests at the very beginning 
would be helpful— it came out later but could have been presented first. 

I did feel some of it was a bit boring. 

Even though the workshop is for non-educators, relate the "technical 
jargon" to laymen's tem^s, i.e., use norm referenced and continue to 
give your excellent examples and explanations. 



Session #3 (n= 5) 
Seems excellent 

I think most aspects were well covered. Four of the five questions I asked 
were answered. 

More time allotted for discussion of each question. 
None that I could see. 

Dealing with teachers, teaching methods. Keeping children in groups 
without letting them move ahead as they progress. 
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Seminar for Parents 

April 12, 1980 
Baltimore, Maryland 

What did you like most about the workshop? Why? (Total n=51) 
Session #1 (n=28) 

I thought it was very interesting. 

All the information was helpful because it was what I needed to know. 

All the information was geared to what parents are really trying to find 
answers about the testing of their children. 

The manner in which it was carried through. The handout that was passed out. 
Alot of contact was between parents and relationships in schools and testing. 
Everything. All of the workshop was important. 
How informative you were. 

You try to answer all the questions, and good job all through. 
The questions were answered. 

We had a chance to find out why they use them and how we as parents 
could help our child. 

Clear up many questions. 

The workshop helped me to understand about the testing. 

The organization of the workshop. Taking in consideration the questions 
the participants wanted to discuss (checklist). The simple way the 
questions were handled and answered. 

Outline the questions that were most important then trying to cover 
topics that could have been covered later or from another source. Time 
concept very good; so often you get side-tracked and can't cover nearly 
enough of researched materials you prepared. 

Discussion period. There was alot of participation from the group. This 
was not a cut and dry workshop. 

The form of the presentation. 
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staff and wide range of information. 

It made me more aware of what type questions I should ask and the persons 
I should address them to. 

I liked the workshop because it gave everyone time to express their 
different viewpoints, to ask ways of improving the environment of testing 
and things that could express ways of lining up a child's test scores. 

The open discussion and answer to parents' concerns regarding tests. 

You need more time. 

That the presenters did not talk down to us. 

It was very informal and most questions were answered. 

Handout and visual aids. 

Debate about testing. 

Handout. 

Nothing, no help at all. 
Nothing, no help at all. 

Session #2 (n=12) 

Learned what questions to ask. 

Did not know much about test scoring. It was explained very well. 
Simplifying the different kinds of tests. 
Questions were asked and you answered them as best you could. 
Group discussion—interaction. 

Working from people's concc 'ns. Good teaching technique anywhere. 
The way they presented the workshop. 

The directness of the speakers and the concerns they showed. 
My questions were answered and I had a chance to talk. 
Open discussion. 

Manner in presentation; group participation. 
The presenters were diplomatic. 
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Session #3 (n=1l) 

Excellent questions we've been asking and dealing with at home. 

The idea that scores are not the most important aspect of testing. 

Workshopees have the choice of questions answered. Made the time I 
spent here most worthwhile. 

Excellent information— resource informant gave good topics to discuss. 
The opportunity for parents to choose topics for discussion. 
Explanation of topics. 

Parents knowing more about the tests given in schools. 
Time for interaction; visuals were good. 
The discussion and explanation. 

It was stressed that we should not hesitate to question the schools* 
testing rules. 

The exchange of ideas. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for Parents 

April 12, 1980 
Baltimore, Maryland 

Do you think others would find the workshop helpful. Why? (Total n=41) 

Session #1 (n=2l yes; 2 no) 

Yes. 

Yes, it is very educational and helpful. 

Yes, because when it comes to the parent, teacher and children there is 
always help needed at all times. 

Because testing is being done in so many ways and styles that parents 
need to know more to meet their childrens' needs. 

Yes, it has alot of information. 

Yes, because alot of people don't know how to help their children in this 
area. 

Yes, because we want the same thing for our children. 
Yes, because it is informative. 

Yes, everyone would. It's hard to understand why test scores can drop or 
rise so rapidly from one testing period to another. It is also helpful 
to know how to relax anyone for testing. 

Yes. Enable them as parents to see how effective the testing material 
is and whether change is needed. 

Yes. Some other parents may have a child in a similar situation and has not 
already found an answer. 

Parents do not know enough about testing. 

Yes. Most parents are concerned about testing in the schools. 
Yes. That will give parents ways of improving amount of testing. 
Most definitely; it has an interest to all parents. 
Very much so, I just wish that you had more time. 
Yes. This issue should be understood by all parents. 
Q I do if they are given the time and are good listeners. 
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Yes, tests are a real concern of parents. 
If you do the talking and parents listen. 
Yes, discussion was good, but not long enough. 
No. 
No. 

Session #2 (n=9 yes; 0 no) 
Yes, was very simple and clear. 

Yes, because the way and reason for the test and the difference in the 
national and local. 

Yes, because most parents are not aware of them. 

Yes, we can all keep learning— there are so many misconceptions that people 
need help with. 

Yes, because alot of parents don't understand tests. 

Yes, if they are totally upset about tests and their effect on performance. 

I think others would find this workshop very helpful because so many do 
not know about testing in full and the value of it. 

Yes; very informative. 

Yes. Important information. 

Session #3 {n=16 yes; 0 no) 

Yesl Parents need to know they are on the right track 1 Thanks 1 

Yesi Parents need to understand testing to better deal with it at home. 

Yes, give them something to take back to their districts and to check 
their testing programs. 

Yes, parents need to become more aware of testing in the schools. 

Yes, it will give parents a better explanation of testing and that they 
should ask questions of the school. 

Yes, covered the purpose of testing. 

That's hard to say about other people. 

Yes, parents, like myself, need to be reassured that they can question 
how their school is being run. 

ERXC" ''^^ people know why and how tests are used. 



Yes. 
Yes. 

Yes, good infonnation. 

Yes. 

Yes! 

Yes. 

Yes. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for Legislators 

June 18, 1980 
Columbus, Ohio 



What changes do you think we should make in the workshop? (For example , 
ether points that should be covered, points that were unclear, or other 
styles of presentation J (n=13) 

The presentation seemed to be made with the underlying assumption that minimum 
competency testing, in some fo'^^t, is worthwhile. Maybe the philosophical question 
as to whether they are necessary at all should be raised. 

The constant reminder of the time constraints seemed to hinder a great deal of 
conversation and inquiry. It would be useful to have a handout of materials presented 
on the overhead projector for future reference. 

Allotment of specific times for questions. Touching on just one issue during such 
a short time span. 

I wouldn't make any major changes in the workshop. However, i.t may be desirable 
to drop the word "psychometric" from your presentation, as few in the audience will 
know its meaning. I would like to commend you for not using educational testing 
patois, with the aforementioned exception noted. 

Probably more material than time allowed. 

None. 

Depending on the audience, more time could be left for questions and clarification. 
Methods by which we could relay information to our constituents, other than the 
extensive handbook. 

Perhaps the weighting of time allotted to each topic could be somewhat more 
flexible (unless of course the "voting" showed the 5 topics to be of about 
equal interest) 

Tried to cover too much content in too short a period of time. 
More detail might have been useful — like cost comparisons. 

Allow more time for questions and group discussion. Provide samples of legislation 
enacted or proposed in other states. 

Hand out reproductions of overlays — it*s good info to have and it's not in 
the booklet! 

It would be helpful to have examples of statutory language (adopted-proposed) 
from other states available as you refer to them. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for Legislators 

June 18» 1980 
Columbus » Ohio 

What did you like most about the workshop? Why? (n=14) 

It was well organized, well presented and moved quickly. The presenters worked 
well with the audience. 

Presentations were well-organized and concise. The informal atmosphere aided 
the discussion. 

The information was very clearly given. .The presenters were obviously well 
informed on their specific topics and in fact could have given a seminar on any 
one of the topic areas. 

The summary of the California experience with minimum competency testing. 
Although some of the problems encountered by California are probably peculiar to 
California (the large # of non-English speaking students), many of the same 
problems encountered by California will probably be faced here. It is helpful 
to know what difficulties we may face in developing a minimum competency testing 
program in Ohio. 

Generalized overview — this is what legislators needs. 
Ran on time ~ whetted appetite to get more info. 
Up to date info and statistics on testing. 

The covering of a variety of information in a short period of time. 

Permitting the audience to direct the focus (tailor-made to a degree) 
Interaction with the audience. 

There was a wide variety of views from outside our state. We spend too much 
time talking to ourselves and need fresh viewpoints. 

The key issues were presented in an efficient and summary manner. 

Informality. Concise presentation. 

Informal enough to engender discussion. 

I welcomed the broad stress on the policy implications of legisltive decisions. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for Legislators 

June 18, 1980 
. Columbus, Ohio 



Do you think others would find the workshop helpful? Why ? (n=12 yes; 0 no) 

Yes. It is a topic that ultimately affects all aspects -of the community. 

Definitely. Extremely informative considering the time limit. 

Yes. Parents and anyone interested in the vital area of testing in our 
educational system. The major reason being the.- amount of confusion expressed 
by much of the public sector in this area. 

I think the workshop should be given for teachers who may well be, as a 
group, the biggest antagonists of minimum competency testing. 

Extremely. 

Issues are clearly laid out— appreciation of the complexity of the problems is 
most helpful . 

Yes, broaden knowledge about testing. 

Yes, content is relevant, well presented and timely. Thanks 1 
Yes, very well done I 

Yes~-content of workshop is contemporary and important. (Information 
is often dated by the time it becomes available from other sources.) 

Yes, the key issues were presented in an efficient and summary manner. 

Yes, competency testing is a buring issue with most people today. Many 
educators (and legislators!) are opposed to the idea. It is helpful to 
find out that competency testing programs are working in other statesl 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for School Board Members 

June 20. 1980 
Columbus. Ohio 



What changes do you think we should make in the wrkshop? (For example . 
other points that should be covered, points that were unclear, or other 
styles of presentation. ) (n=54) 

Too many educators in the audience — A lot of Hie terminology went over the heads 
of board members — Not the fault of presentors ~ They simply responded to questions 
— unfortunately a lot of questions were from persons other than board members 
I thought it went well considering the diversity of the group. 

What are the specific goals of the conference? i.e., information input or output? 
Would like the names, positions of leaders. What is NIE. NWREL. etc.? 
Bibliography of books, cases discussed. More handouts. 

Clarify the simulation — Do you want the group to act as a Board (policy making) 
or a team of administrators (who will make specific recommendations)? 
Don't make the introductions too long. 

It is quite good as is. 

There should have been more handouts. 

Attempt to allow time for a person to take in both of the concurrent sessions. 

A time set aside for a State Dept. representative to update the group on current 
state action. 

I wanted to participate in simulation and issues discussion — Could both happen? 

Being a layman, I would have liked a hand-out listing tests commonly given for 
different purposes & their strengthes & weaknesses. 

You should have presented in written form a listing of key problems and subsequent 
recommendations typically reported for simulation session. 

Suggest that simulation session have more of a purpose & pay-off for participants. 
It is difficult to see advantages of this approach. 

More time for the 10:30 - 12:00 items especially on the competency testing. 
Should form smaller groups, after main presentation. 

Clearer definition or discussion between standardized testing and teacher made tests. 

Smaller groups initially. More discussion time re competency testing (student 
and teacher). 
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Might have shown examples of competency tests both local products and commercially 
drawn up types. 

Later start. Provide writing materials for note taking. 

Points covered were covered well. I would like to see more emphasis upon how one 
reports data to a Board and how can the Board interpret the data. The model used 
during the session was passed over too quickly. 

Statewide workshops should end sooner. 

Nofic. (Maybe more material available or-order forms) 

A presentation by a lawyer who is conversant on testing would take away the "I'm 
not sure but..." or "I am not a lawyer..." statements. It would not have to be 
a long presentation, but maybe as supplemental or just present to assist in 
answering questions. Based on the morning presentation more time should be 
allotted to the Major Legal Issues. There seemed to be a great deal of 
interest in these areas and time was short at the end of the morning 
session. 

I feel that the workshop was very good. I feel that regardless of the "height*' of 
standards that colleges and universities create for teacher graduates (requirement 
wise) etc... a workshop of this type should emphasize that application on the part 
of the teacher in the classroom is more important. Examples: 1) The best athlete 
doesn't always make the best coach. 2) The most intelligent student doesn't 
necessarily make the best teacher. 

Smaller groupings for better exchange of ideas. 

Good job. 

Some examples of tests used should have been passed out and commented on. Many 
school board members, attorneys, etc. have never seen an ability or an achievement 
test. At least a few sample questions would have helped. We spend a day discussing 
tests and never see one. 

No suggestions. 

The people who attended were from mixed backgrounds —some were directors of testing 
— others knew nothing about testing and all wanting very simplistic answers to 
very difficult problems — I think you did a good job of meeting the needs of some 
of those present — 

Most points were clear. The final rap session could avoided, because during 
the particular breakdown group I attended this was already accomplished. 

Presentation was excellent. Should be broken down for more time in each area. 
Should be at least two days. 
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Simulation on testing — you ask the board to make administrative decisions 
when as a board they should merely set policy. The task should be given to a 
group of administrators. OR the Board should look for new administrators of 
the district. Goals should match the role and function of the group. 

I wonder if it would be possible to discuss specific tests and their purposes 
(by name) Are some really better than others? Costs involved in administering, 
grading, doing item analysis? 

hore help to build a good program. 

Too much repetition uf maLerial. The approach' to competency testing v;aG biased 
against this type of testing. 

More sequential (developmental) presentation and greater use of A-V aides may 
be an improvement. 

Might have more (specific) handouts on each topic; perhaps more visuals; need 
names of presenters printed on agenda. 



s 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for School Board Members 

June 20. 1980 
Columbus, Ohio 

What did you like most about the workshop? Why ? (n«=39) 

Smaller sessions - gave more people an opportunity to ask questions etc. 
I learned a lot about trends in other states, including litigation. 

Informal presentations. 

Gordon's presentations -~ very fluent and proficient in his subject. 
The high level of expertise forced me to reach up! 

All presenters are quite evidently intelligent people who are also articulate. 
The breadth of presentation will enable me to find sources for more depth in 
particular areas of testing. 

It served as an overview. 

Knowledgeable leaders. 

Clear summaries. Clean categorizations. 

Introduction to purposes, uses, of test scores. Introduction to legal issues 
in testing. Being an attorney, I never received training in educational 
testing. ..this was very helpful to me. 

Enjoyed the clear - easy to understand concepts/terms. The support material 
packet is appreciated. 

Well organized — open for questions during the topics. 
Talking to the presentors at breaks. 

Current issues — small groups frequently more stimulating and effective. Presenta- 
tions were generally geared to the level of nonprofessionals which was appreciated 
as a board member. Kept to the schedule — always an asset. 

Information that is useful in developing a district testing program in relation 
to identified programmatic objective. Pitfalls most helpful. 

I liked the discussion group best because discussion nearly always opens wider 
vistas of understanding and clarification. On second thought I think I like 
your handbook best. Not only does it cover most of your presentation but I also 
have something to take home to refer back to in discussions with others in rny 
district. 

The expertise of the presenters and the organization of the day. It was fast moving 
yet not rushed. 

Discussion centering on competency. This reflected the greatest area of interest 
for this group. 
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Small group interaction. A group project that was well thought out. The summary 
— input from each group plus additional points overlooked. 

The variety of presentations and the group participation. 

High interest topic. Good presenters. Impressed with , expertise of presenters. 

The many facets on testing were presented and discussed. The free exchange of 
ideas by all present was very encouraging. 

Give and take — • freedom to ask questions — make comments. 

The open atmosphere of presentation and receptivenessof opinions is gratifying. 
Specifically, the time allotted to questions and answers was excellent — especially 
since the emphasis was on answering the questions, not covering a specific amount 
of material. People could get their questions answered and not left "up in the air." 

General info. 

Concerns of testing and how results are used. Because of the practical application 
that is needed. 

The section that delineated the legal ramifications of testing. 

A good general coverage of the topic in layman's terms with expertise demonstrated 
where probed. 

The speaker with the beard was the most interesting. His optimism about tests 
was a welcome change. The attorney in the gray suit was helpful on legal issues. 

I guess to discover that all of you are having the same frustrations as I and 
testing is not going to solve the problems of education — certainly not with 
competency testing. 

The way the agenda was presented. Clear and precise, moving along at a steady pace. 
Informal presentation. 

The confusing ideas that I developed. My opinion of testing, I find, was very 
narrow. 

Overview at beginning. 
Personal involvement. 

Speakers were well- informed and presented themselves in a manner which indicated this. 
Very informal which lends itself to more questions and comments which is a benefit 
to all. 
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Current Issues. The sumnarlzatlon was good. 

Speakers did a nice job of presenting and responding to questions; content was 
In-depth, but not too technical; content was appropriate; whole day was nicely 
paced. 

Format was good — time well organized — presenters knowledgeable. 
Most of the discussion. 

Enabled me to acquire some knowledge regarding testing approaches. 
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SUMMARY OF PART I C ^ ' in I REACTION S 



Seminar for School Board Members 

June 20. 1980 
Columbus. Ohio 



Do you think others would find the workshop helpful? Why? (n«34 yes; 3 no) 

Yes, we get so involved in our little world that we don't know what's going 
on elsewhere— I think it's good to know the national trends. 

Yes, focuses on problems— need more guidelines on i«hat a good test program 
might look like. 

Yes. Overview of testing, including legal issues, is presented simply and 
clearly. Good for Board members. 

I think that more teachers should have attended the seminar. I am a 
teacher and have enjoyed not only the presentations but also the company 
of members of what is--often— the hostile side, at least from some 
teachers' points of view. Each segment of the education field needs to get 
together with each other segment. 

Yes, because of the information discussed. 

Yes, parent groups— knowledge gained about subject. 

Yes, informative, clear, concise. 

There is so much rryth and misconceptions associated with testing that it 
is necessary for those who are not experts to have a knowledge of what 
tests can and cannot do. 

Yes. Presented in very understandable terms. 

Yes. Very few people are not interested in their children— very few 
understand the reason behind testing. 

Yes, all Board Members in my county— they need help. 

Yes, good overview— some idea of current problems and directions in the future. 

Most board members are not "professionals'' and have little of the background 
and knowledge presented unless they happen to have an administrator 
particularly interested in the subject. 

Yes, especially if districts sent teams to resolve specific district 
problems. 

If it were possible to broaden the base of attendees to include teachers, 
a better understanding of testing might be possible. 
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Yes, especially lay people with Interest in the schools. 

Yes. The competency issue needs to be discussed and the pros-cons of 
this testing concept. 

Yes, the issues are current and pressing. 

Yes, because the session has been very informative and comprehensive. 

It has focused upon the most important points. Provide writing materials. 

Yes, the workshop can serve as a primer in testing and other forms of 
evaluation. 

Yes, all school people are interested in this topic. 

Very much so. Most of us are biased on what we think testing can and 
should accomplish. 

Yes— good inservice type program given by presenters who have obviously 
done their homework and been through the mill. 

Yes— too many (administrators, boards, etc.) see testing as an entity and 
a separate item unrelated to the whole curriculum. Also, some (administrators 
and boards) see testing as just saying "We give all these tests, therefore 
we are doing a good job*' etc. Many of the decision makers are not competent 
or possibly totally ignorant in the field of testing. If nothing else, the 
people in positions of authority could talk conversantly about testing and 
not be "bumps." 

Yes— provides some answers. 

Yes, to better meet the needs of "all" students. 

Yes, informative. Booklet contains valuable information. Lots of reference 
points/areas given. 

Yes, a good general coverage of the topic in layman's terms with expertise 
demonstrated where probed. 

The legal discussion might be helpful. Otherwise, no— not helpful. 
Wasn't much of a help to me. 

Considering the different groups represented, board members, teachers, 
administrators, etc. No. The final decision still rests upon our educators 
using testing as a guide and not a label. Our society will always test, the 
content is what must be considered and not the final grade. 

Yes. 

Definitely. 
Absolutely. 
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Very definitely. 
Yes. 

Yes, to some extent. 



SUMMARY OF PARTICIPANT REACTIONS 



Seminar for School Board Members and Journalists 

July 12, 1980 
San Francisco, California 



What changes do you think we should make in the workshop? (For example , 
other points that should be covered, points that were unclear, or other 
styles of presentation J (n«16; 

More time on simulation — more of how to choose tests appropriate to what 
might be needed In public schools. 

Might have been fun to take a short sample test* It's been a long time for a 
lot of people. 

Some of areas covered in discussion period repeated in future trends out of necessity 
— for board members. Suggest consolidate afternoon sessions to eliminate that. 

If not too expensive , a 1 page sheet of state laws concerning testing (or the 

issue at hand) to be used as a reference by participants. 

If not too expensive , the handbook, sent before the meeting would be helpful. 

At least mention sex discrimination many people have little awareness of this bias. 
Use black flow pen. 



Examples of tests used would be helpful. Perhaps threatening that we might be tested 
to get a feel of it (after so many years away) 

Maybe shorten the number of topics, that would allow participants to linger in 
areas of high interest rather than to fulfill an agenda that may be too long. 

Simulation exercise v^ry good but really needs more time for reflection to glean a few 
more points. 

A more succinct statement of issue (generally this was done but there were some 
problems) and though there should be leeway for question and answer (In many cases 
the best part of the program), when we begin straying from the topic, someone 
should quickly put us back on track. 

How about a section on coaching for tests pro and con ~ 

A list of resource material i.e. a bibliography for us to follow up on (possibly 

along with the table of material) 

Larger print on name tags. As an aid to communication, they must be easily read. 

Go around the room with introductions. Again group gains from knowing who is present. 

I don't think I could absorb any more. 

I would have preferred more time be on cultural bias in testing. 
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Effectiveness of tests in determining future failure/success. What 
competencies should be tested. Relationship of competency testing to 
employer required skills. 

How do we use test results to diagnose what we really need to be doing 
to help children do better? What do tests really tell us? How do we 
know what tests are most useful? or. ..Who evaluates tests? 

Could some material be sent out in preparation? I.e., the Facts & Issues 
booklet. I found many questions popping up based on rr\y ignorance. 

Test samples might be helpful. More use of A.V. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for School Board Members and Journalists 

July 12, 1980 
San Francisco, California 

What did you like most about the workshop? Why ? (n«18) 



The presentation of both sides of issues. 

Explanation and more understanding of testing. Lay persons (board members) 
often don't know the proper questions to ask. Tm anxious to read the handbook! 

Covered a great deal in a short time and did a good job. 

Strongest aspects were legal issues and discussion session. Felt you touched 
on the topics journalists, especially, are interested in and did not clutter 
up workshop with minor issues and too much detail. 

Dave Gordon's presentation was extrememly clear. Granted, he followed the booklet, 

but his verbal explanations were clear, explicit and to the point. I 

appreciated the absence of jargon. 

The "guide" as a background is a good reference. 

Overall organization, staying to schedule. 

The intellectual stimulation. The information. The contacts - meeting others. 
The references - the booklet. 

Discussion of future - rapid change will make much of the past and present obsolete. 

The small group setting. It enables more questions and answers. 

I obtained a better appreciation of Board members' viewpoints. 

The discussions between school board members and reporters were fascinating 
both media and board members had a good overview but intriguingly different 
perspectives on the issues — made for lively debate/conversation. 

Bev's first-off newsprint listing of group's suggestions. It was great to know 

we weren't going to have to be passive all day. 

Booklet. Hope it contains enough. 

Round tables at lunch and during meetings. Friendly. 

General excellent level of the presentation. Thank you. 

The opportunity to interact. Nice to have an entire day to concentrate on one 
subject in depth. 

Simulation activity offered an opportunity to interact in small group recapping 
and expanding on most of the items covered during morning session. I felt this 
session was well placed after lunch. Nice change of pace. 

Current issues in educational testing. 
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Discussion. Legal Issues with testing presentation. 

Super job ~ keep the open discussion format. I don't give up rny Saturdays easily (or 
happily) but I found this workshop OUTSTANDING, It did more than answer questions. 
It raised issues and helped me become aware of what I need to know, what questions 
need to be raised in my district. Good agenda - neatly stated and followed . 
I liked the "future trends" discussion. 

Interaction within the group, (opportunity to exchange ideas). Presentations 
were clear (Information was useful.) Orderly manner of presentation/s (Inspired 
confidence that you "knew your stuff".) Presenters were well prepared. Good handouts 

Casual atmosphere. Opportunity for audience participation. Well organized 
agenda. (Excellent luncheon!) 
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SUWARY OF PARTICIPANT REACTIONS 



Seminar for School Board Members and Journalists 

July 12, 1980 
San Francisco, California 



Do you think others would find the workshop helpful? Why? (n=18 yes; 0 no) 
Yes. A lot of people need this sort of information. 
Yes—Board members should know how to interpret the materials they read. 
Yes— it was for me informative. 

Yes. Others interested in i t--teachers, administrators, reporters. 

Yes— especially administrators and teachers. Could ACSA and CTA and 
the State Department cosponsor area meetings similar to today's session? 

Yes— all Board Members need this as part of orientation to job of school 
board. 

Yes. Good background material. 
Yes. Good basic information. 

Yes, parents, PTA boards, citizen advocacy groups in education, even 
students (student body presidents, student councils) are all consumers 
of the tests— and could benefit from a similar workshop, possibly retooled 
slightly to meet their special needs. 

Yes, perhaps the booklet will be enough. Perhaps a summary of the booklet 
would be worth the effort. It is very hard for busy people to give up 
a Saturday, even though the material is of great interest. 

More board members should avail themselves of this opportunity. 

Yes. As we get more involved in the testing program they will need to 
understand why we have these programs. 

Yes. The workshop was geared for school board members which is not often 
the case. Since many board members have other commitments, Saturday was an 
excellent choice of days. Handouts were good. 

Yes. It is practical, realistic and informative. 

Yes, but they need to get more information about what and when and why. 
Our staff people involved with developing competency testing could've found 
this useful. 
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Yes, As Board Members we are pressured to leave the issue of testing to the 
"professionals" (educators and administrators). This should be encouraged 
for one board member from each district. 

Yes, It provides an opportunity to clarify tests/results to the lay person; 
the reasons for testing are examined 5 means to improve programs are discussed. 
Positive results may be possible when you return to your own district. 

Yes. Current relevant information. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for Lawyers 

September 25, 1980 
Oakland, California 



What changes do you think we should make in the workshop? (For example, 
other points that should be covered, points that were unclear, or other 
styles of presentation.) (n«21) ~ 

As an advocate I am most interested in learning the weaknesses and strengths 
of various types of standard tests so that I can assess their usefulness or 
plan methods of attacking their validity under specific circumstances. 

More specific information on testing for handicapped children would have 
been more helpful for me. 

Quicker introduction to basic concepts and extended simulation, or other activity 
which requires participation. 

More time should be allotted for simulated activity regarding interpretation 
of test results. More detailed explanations on how to interpret test results 
would be immensely helpful for those who have no prior experience in 
interpreting tests. In addition, information on how to legally address 
cultural bias In testing, rather than simply explanation provided in workshop 
on cultural bias and I.Q. tests would be valuable. 

Provide more basic skills and overview for non-lawyers. How to Interpret 
test results for students and parents, how to evaluate their overall results 
for the students and how to make recommendations pertinent to the students* 
needs in an education plan. 

More analysis, more research, a more detailed factual presentation. 

Some of the presentations were a little too basic. Book on a Parent's 
Guide to Testing in the Schools is very poor. Should point to abuses as 
well. 

Morning overview was not a good use of time— discussions regarding the 
construction and use of test Instruments would be better. General overview 
could be sent out in writing before workshops. Test purposes, uses and 
scores did not contain what it was represented to be. 

Testing simulation somewhat too technical; afternoon workshops better. 

More in-depth understanding of tests themselves and what they are supposed 
to measure. 

More information on specific tests. 
More materials. 
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Presentation on purposes, uses, scores was too pedantic—not particularly 
helpful. Time would have been better used to expand other sessions. 
Workshop time too short. Simulation activity useful in terms of information, 
but format was simplistic (it was not a true simulation). 

The simulation activity was unclear and should be eliminated. I would 
suggest three sessions in the afternoon instead of two. 

Cover psychological testing. 

Simulation exercise was not worthwhile. Inadequate delivery of information— 
not enough delivered. 

Preliminary lecture was too overbroad and simplistic. Not useful. It could 
be more helpful to discuss various tests, their content, purposes and other 
relevant points. Simulation activity not well thought out. Without adequate 
preliminary, it was not do-able by trainers. . 

Include Black and/or other third world trailers. Be more vigorous in 
cautioning trainees regarding the dangers of standardized testing--especially 
in the Parent's Guide. 

A concise review of the commonly used tests with criticism and an 
explanation of the purposes and indications of the test (e.g., WISC-R, 
WRAT, KEYMATH, etc.) 

Simulation model too complex, not enough prior background— we discusssd this 
fairly thoroughly in our session. Parent booklet ignores most important 
issue that tests are only limited tools which can be useful or very damaging 
depending on how they are used. Even if parents do not understand technical 
details of test, they certainly can understand the various potential and 
actual uses and misuses of test results. The booklet tends to justify testing 
as a good and useful thing without critical analysis. 

On Cultural Bias in Testing--simply more time for questions, etc. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for Lawyers 

September 25, 1980 
Oakland, California 

What did you like most about the workshop? Why? (n-25) 

Listening to the opinions of other participants. 

The simulation activity was very good in that it provided some direction 
for practical application. 

Competency testing workshop— the discussion dealt with the types of 
litigation issues J am interested in. . 

Discussion of minimum competency testing. 

Cultural bias workshop. As a result of its practicality and ability to 

convey to all folk present an understanding of cultural bias in tests, validity 

and use. 

The hypothetical was good. Easy to understand. The presenter had viewed 
different issues. 

Good introduction to many issues. 

The concept was good but the delivery broke down. 

Materials with general guidance into more in-depth materials in relation 
to education programs. 

Cultural bias workshop- -"Normative, technical enough and interesting. 
Presentations— trainers were prepared* 

The format— group— small workshop was good for process of information, 
however more continuity of program is desirable. 

Presentations were geared to inform exactly what they were designed to inform. 
Competency workshop. 

Session on competency testing— very specific and well-organized. 

Well done. Excellent deliverance and totally comprehensible. Instructors 
should be commended for their excellent preparedness and deliverance. 

Excellent information; generally well presented (with exception of opening 
session). Sessions flowed well. As CL&E trainer for previous four days, 
I was exhausted, but found this surprisingly stimulating. 
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The workshop on competency testing was very interesting for the following reasons: 
the leaders were well prepared and the group actively participated in the 
workshop. 

Explanation of test scoring, norms, and checking of validity of tests. 

Diana Pullin's talk about legal challenges to minimum competency testing 
was most useful for my purposes. 

The workshop on Parents, 

Pretty good session on Cultural Bias. Helped put the issues in perspective 
conceptually. 

The simulation--most practical, "doing"-type exercises of conversion, 
chance rate, etc. 

Bias session and competency session--cohveyed most direct and useful 
information and analysis. 

The group leader took time to make materials and presentation understandable. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for Lawyers 

September 25, 1980 
Oakland, California 



Do you think others would find the workshop helpful? Why? (n=22 yes; 1 no) 

Yes, it provides information essential for parents as well as people working 
on testing issues. 

Yes, most advocates simply lack the know-how to assess the tests given by 
the school systems in their localities. Experts in testing are rarely 
consulted prior to the initiation of litigation because of cost factors, etc. 

Yes, it is rarely provided and many folk are advocates in the area of 
education. 

res, racial bias in testing needs to be thoroughly researched in order to 
ensure that everyone gets an excellent educational opportunity. 

Good for persons needing an introduction into the area of testing. 

Yes, but in the context of the LSC conference it should have been given 
earlier. 

Yes, all looking for general guidance for education process materials. 

Yes, if more time to discuss various strengrths and weaknesses of tests. 

Yes, depending on experience— good review of techniques. 

A similar workshop directed to parents would be very valuable in 
raising their consciousness as informed consumers and help to clarify 
their role in the education process. 

Yes, informational aspect. 

Yes, it is an area that we know so little about and it's so important. 

Teachers, counselors and administrators should find it helpful if they're 
open-minded. 

Most certainly. The information is enlightening, and extremely helpful 
to attorneys, parents and paralegals. 

Yes, good information— generally good presentation. 

Yes, it provides a good introduction in the area of testing. 
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Sure, much of school programming is based on testing and an understanding 
of testing is crucial to an understanding of American public education. 

Not attorneys or professions. Actual instruction was pretty mickey-mouse 
and paternalistic/patronizing. Most of audience was more well informed 
than lecturers expected. This workshop would probably be more useful 
for parents, students. 

Marginally. 

Yes, because very few people, including teachers and educational 
administrators really know how to critique and apply the tests. 

Yes, however, participants need fair bit of existing knowledge about 
testing in the schools. 

Yes, the group leaders took time to make materials and presentation 
understandable. 

More time for simulation would have been helpful. 
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SUMMARY OF PARTICIPAIT REACTIONS 



Seminar for LaMQfers 

September 25, 1980 . 
Oakland, California 



Did you find the morning test information gathering simulation to be 
Instructive? Why or why not? (n«Z3) ~ 

Yes, the simulation provided some direction for practical application. 

Yes, the participatory method allowed for a better chance to understand 
the testing procedure, validity, etc. 

Yes, most practical. 

Yes, in terms of information and methods of analyzing tests. No, in 
regards to simulation. In terms of personal training through National 
Training Lab, this simulation was questionable training strategy. 

Yes, helps us to understand how scores are read. 

Yes, it provided a better starting point for utilization of expert 
witnesses, i.e., knowing what to ask about> etc. 

Yes, especially Part I. I wish Part II Mould be a lecture type situation. 
Yes, showed how to determine ranking. 

Yes, but not enough time— would have been better to dispense with preceeding 
lecture session and spend time on this. Also, hypothetical should have 
clearly stated that cultural bias was not an issue. 

Yes. 

Yes. 

Yes, the simulation indicated that parents and lawyers can understand and 
make informed decisions with the proper information; that there is no need 
to be intimidated by education professionals. 

This session brought home the point that you need to look beyond the ultimate 
"result" to determine the appropriateness of the test to a particular 
situation. I lacked sufficient understanding of various tests and test 
jargon to evaluate the tests used in Task 2. 

It was instructive, however, there was not enough "tying together" of the 
exercise and reality in applying the skills. 

I learned how to read tables and convert scores for purposes of determining 
the chance score problem. I get the feeling that there is more involved 
in this area however. 
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Parti y--there was not enough time or lecture on the tests we were 
evaluating. There was an unfortunate presumption that this is very 
difficult information and you probably can't grasp it. I don't know 
if this was intentional, but I certainly don't agree. More time should 
have been taken to discuss basics. 

Somewhat— if not used, information will be forgotten. Too many tests 
and variables in too little time. 

More time necessary. 

Not familiar with education process enough for the lev,el which the 
simulation situation called for. 

Simulation model too complex, not enough prior background. 

Simulation exercise was not worthwhile. Inadequate delivery of information. 
Net enough delivered. 

No, too much geared to educators. 

No. 
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SUMMARY OF PARTICIPANT REACTIONS 



Seminar for Lawyers 

September 25, 1980 
Oakland, California 

Please indicate those concurrent workshops you attended in the afternoon 
and comment on their effectiveness . 

Helping parents and children cope with testing {n=7 ) 

Small group permitted good exchange of ideas and opinions. 

Very good because of the small size which alTowed for a lot of discussion 
and exchange of materials. 

Good chance for informal discussion and infbnnation sharing. 

Not real effective— insufficient structure and failure to discuss the 
political problems attached to the misuse of standardized tests. 

Did not attend but it appears to have been covered in written materials— 
however, it parrotted testing industry via*s. 

Did not attend but find rnyself seriously omcerned about the "guide to 
testing." You need good quality minority feedback on this piece. I would 
not use it in my parent work. 

The booklet is totally biased. There are aany disadvantages and weaknesses 
in exams. 

Competency Testing (n=15) 
Good 

Interesting. 
O.K. 

Excellent. 
Excellent. 
Fair. 
Good. 

Good discussion of particular interest to ■e-'-perhaps not to others. 



56 



2 



Fine if you were an attorney or a merrtber of an involved competency 
testing group. Worthless for those who wanted an understanding of the 
testing and issues involved. 

Good overview of legal issues involved. 

Presenters were very good. What the workshop lacked was a political 
framework to address why this issue has come up lately and what the 
real purposes of the tests are. 

Good. 

Good. 

Excellent. 

Presenters were excellent but teaching to pass test implicit in that is 
questionable in my view. 

Cultural Bias (n=19) 

Good information and presentation. Liked the style of presenter. Was 
concerned that devious use of tests to validate schools' positions re minority 
and children who are "different" was downplayed. 

Good. 

Good. 

Very interesting and useful, clear. 

Reinforced my pre-concei ved ideas that tests and statistics can and 
will be interpreted many ways. 

One hour is insufficient to even scratch the surface. More indepth of 
morning program might have been more appropriate. 

Practically useless for me at least. Too abstract, not nuts and bolts 
enough for me. Presentation was not very analytical, nor did it encourage 
an analytical approach to determining when bias exists and what the 
instructional and policy implications of that challenge may be. 

A little too abstract but a good introduction to the problems involved 
in the area. 

Gave iDe the sensitivity to cultural bias necessary to advocate on behalf 
of students and groups being tested. I did not agree with all of the 
presenter's interpretations, but her premises and theories were sound. 

It was interesting to get a "technician's" view of tests. 
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A good explanation, simplifying the process of analysis for bias in 
terms everyone can understand. One reservation: I am not sure I can 
agree with the statements that (1) boys throw balls better than girls, 
or (2) that is is not a hot, political issue. 

Fair. 

Good. 

Good discussion. 

So-so. Too theoretical and not at all geared toward practical uses. 

Presented in a most understandable fashion; not just in dispensing 
information, but in highlighting difficult areas. 

Pretty good— should have developed a more politically focused understanding 
of the issue. But did a good job of developing a conceptual framework 
useful to understand traditional means to measure cultural bias. 

Poor— as a trainer in the LSC program, I thought the presentation was 
not well done. Appeared to be a defense of testing industry. 

Good. 



